test(v2): live contract tests for all 5 client.v2 endpoints#109
Conversation
Add tests/contract/test_v2_smoke.py covering files.upload, extract (sync), extract_jobs (create+wait), parse (sync), and parse_jobs (create+wait) against live staging, mirroring test_v1_smoke.py conventions.
There was a problem hiding this comment.
Pull request overview
Adds live staging contract coverage for all five synchronous and job-based V2 client surfaces.
Changes:
- Adds five V2 contract smoke tests.
- Adds a PDF fixture for parse tests.
- Uses bounded polling and skips when staging credentials are unavailable.
Reviewed changes
Copilot reviewed 1 out of 2 changed files in this pull request and generated 1 comment.
| File | Description |
|---|---|
tests/contract/test_v2_smoke.py |
Exercises V2 upload, extract, and parse APIs. |
tests/contract/sample.pdf |
Provides parse-test input. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
…review) Co-Authored-By: Claude Opus 4.8 (1M context) <[email protected]>
…hanges griffe flags a changed value of a public constant (e.g. adding staging/dev to the ENVIRONMENTS map) as breaking, tripping the gate on every PR since #105. Additive config is not an API-signature break, so filter griffe's 'Attribute value was changed' breakages; any other breakage still fails the gate (verified: renaming a public method still trips it). Removes the need for the breaking-change-approved override on such PRs. Co-Authored-By: Claude Opus 4.8 (1M context) <[email protected]>
Update: live-verified + surface-lock gate fix folded inLive verification (staging is back): ran the full contract suite against staging — 6/6 pass (1 V1 + 5 V2: files.upload, extract sync, extract_jobs, parse sync, parse_jobs incl. the new result-payload assertion). The deferred live check is now green. surface-lock gate fix (293fa2e): per request, this PR now also fixes the gate so it stops tripping every PR on the benign additive Follow-up (not this PR): surface-lock doesn't currently track the V2 resource-method signatures ( |
Correction to the surface-lock coverage note aboveMy earlier note ("surface-lock doesn't track V2 resource-method signatures because they aren't exported from a public The actual reason V2 isn't covered yet is the baseline: |
…ng loop Add a parallel `spec-sync-v2` job that mirrors the proven V1 loop, pointed at the V2 spec on the AIDE gateway (aide.staging.landing.ai/openapi.json) and a separate `spec-sync/v2` branch, with a V2-tailored wiring prompt. Reuses the existing check-drift / fetch-normalize / gen-models scripts unchanged; the V1 job is untouched. The committed baseline `specs/v2-aide.json` is the FULL current spec, so check-drift reports no drift and the loop ships live but quiet — it fires only on a future real V2 spec change. /v2/workflow is intentionally deferred (not shipping yet): it stays in the baseline (so it isn't flagged as drift) and the AI-wiring prompt explicitly excludes the three /v2/workflow* routes so a future unrelated drift can't pull them in. Hosts (do not conflate): V2 spec = aide.[env]; V2 API = api.ade.[env]. The V2 contract tests from #109 gate the spec-sync/v2 branch as the live guardrail. Design: docs/design/v2-spec-sync-extension.md. Co-Authored-By: Claude Opus 4.8 <[email protected]>
…ant AI-wiring) (#110) * feat(spec-sync): track V2 spec drift (AIDE gateway) + dormant AI-wiring loop Add a parallel `spec-sync-v2` job that mirrors the proven V1 loop, pointed at the V2 spec on the AIDE gateway (aide.staging.landing.ai/openapi.json) and a separate `spec-sync/v2` branch, with a V2-tailored wiring prompt. Reuses the existing check-drift / fetch-normalize / gen-models scripts unchanged; the V1 job is untouched. The committed baseline `specs/v2-aide.json` is the FULL current spec, so check-drift reports no drift and the loop ships live but quiet — it fires only on a future real V2 spec change. /v2/workflow is intentionally deferred (not shipping yet): it stays in the baseline (so it isn't flagged as drift) and the AI-wiring prompt explicitly excludes the three /v2/workflow* routes so a future unrelated drift can't pull them in. Hosts (do not conflate): V2 spec = aide.[env]; V2 API = api.ade.[env]. The V2 contract tests from #109 gate the spec-sync/v2 branch as the live guardrail. Design: docs/design/v2-spec-sync-extension.md. Co-Authored-By: Claude Opus 4.8 <[email protected]> * fix(spec-sync): address Copilot review on the V2 loop - Security: drop rye/scripts shell from the AI step's allowedTools (only Edit/Write/Read/Glob/Grep/git-diff) so a prompt-injected spec can't execute code with the push PAT; add a fixed step that discards any agent edit to scripts/.github/specs before running the trusted ./scripts/format. - Broaden the wiring prompt to cover every changed operation backing client.v2 (field changes on existing ops + /v1/files, not only new /v2 routes). - Permit backward-compatible signature additions (new optional kwargs); verified surface-lock does not flag an added optional parameter. - Docs: resources/v2/__init__.py stays unchanged (no sub-resource re-export); genericize the Phase-2 design away from the deferred workflow feature. Follow-up (V1 + V2): persist-credentials:false + read-only action token. Co-Authored-By: Claude Opus 4.8 <[email protected]> * fix(spec-sync): harden V2 loop per second Copilot round - Sanitizer: enforce a product-code ALLOWLIST (src/tests/docs/api.md) instead of a 3-dir deny-list, before format AND staging. A repo-wide agent Write to e.g. pyproject.toml (which defines `rye run format`) can no longer execute in the trusted format step or reach the commit via `git add -A`. Verified in a scratch repo: out-of-scope tracked edits reverted, untracked injections deleted, legitimate src/tests/docs additions kept. - Open-PR check: fail CLOSED — capture `gh pr list` result outside the `if` condition so a transient API/auth failure aborts instead of silently becoming open=false and force-pushing over an existing reviewed PR branch. - PR body: mark the AI commit conditional (workflow-only drift / AI no-op produces a mechanical-only PR), so mechanical-only PRs don't carry an inaccurate summary. Co-Authored-By: Claude Opus 4.8 <[email protected]> * fix(spec-sync): wire component-only V2 drift (response-model fields) Broaden the AI-wiring prompt to inspect changed components.schemas, not just operation/path hunks: a response-field addition changes only a component (e.g. ParseResponse) and its generated model, not the operation. Instruct the agent to trace $ref consumers of changed schemas to non-workflow operations, and to distinguish request fields (new optional method param) from response-model fields (add to the response type under types/v2/). Co-Authored-By: Claude Opus 4.8 <[email protected]> --------- Co-authored-by: Claude Opus 4.8 <[email protected]>
Adds live contract tests for all 5
client.v2endpoints —files.upload,extract(sync),extract_jobs(create+wait),parse(sync),parse_jobs(create+wait) — intests/contract/test_v2_smoke.py(markercontract), plus a small PDF fixture (tests/contract/sample.pdf) for the parse checks.How they run
pr-gates.ymlcontract-testsjob (rye run pytest tests/contract -m contract -n 0) — no workflow change. That job is scoped tospec-sync/*PRs and skips cleanly withoutLANDINGAI_ADE_STAGING_APIKEY(forks/local), so ordinary PRs (like this one) are not gated on live staging.environment="staging"→ V2 routes toapi.ade.staging.landing.ai. Assertions are structural (types/shape/JobStatus.COMPLETED), not exact-value, since it's a live gateway.wait()calls are bounded (timeout=300)../scripts/lintclean (ruff + pyright-strict + mypy).As of 2026-07-12, all
*.staging.landing.aihosts return404on every path (including theopenapi.jsonendpoints that served200on 2026-07-08) — an aide/infra-side change, not a test defect. The pre-existingtest_v1_smoke.pyandexamples/v2_smoke_test.pyfail identically, confirming it's environment-level. The same V2 code passed 5/5 live against staging on 2026-07-08, so these tests are correct by construction; a fresh green run is pending staging recovery. (Separately noted: prod V2 is now live —api.ade.landing.ai/v2/parse→ 401, prod specaide.landing.ai/openapi.json→ 200.)Related: #95 (V2 SDK), the spec-sync
contract-testsjob inpr-gates.yml.🤖 Generated with Claude Code